Abstract Title Page 


Title: 

Project READI: Designing Instruction for Evidence-based Argumentation in the Disciplines 


Authors and Affiliations: 

Susan R. Goldman, University of Illinois at Chicago 
Carol D. Lee, Northwestern University 
Cynthia Greenleaf, WestEd 

Cynthia Shanahan, University of Illinois at Chicago 
Project READI 


SREE Spring 2013 Conference Abstract Template 



Abstract Body. 


Introduction and Background/Context 

Project READI defines reading for understanding as the capacity to engage in evidence- 
based argumentation from multiple text sources of information, situated within a discipline. 
This definition reflects what we see as central competencies needed for success in 21 st 
century society. Yet the evidence indicates that students are graduating from high school 
without these skills (Carnegie Council on Advancing Adolescent Literacy, 2010; Common 
Core State Standards Initiative, 2010; NAEP 2009; OECD, 2004; Biancarosa & Snow, 2004). 
Project READI is developing interventions whose overarching goal is student attainment of 
reading for understanding as we have defined it. We are simultaneously engaged in design- 
based research to develop the intervention and basic studies of the cognitive, social, and 
instructional processes and practices involved in reading for understanding. We are working 
in three disciplines - literature, history, and science - and across grades 6 to 12 (Goldman et 
al„ 2009). 

The development of the intervention is based on a theory of change that posits that students’ 
acquisition of knowledge, skills, and dispositions necessary for reading for understanding 
depends on teachers creating appropriate opportunities to learn. To create these opportunities 
to learn, teachers need to understand the what, why, and how of instruction that supports 
reading for understanding, including tasks, text sets, instructional practices and assessments. 
The Project READI intervention therefore includes (1) the development of exemplar 
instruction in the form of Evidence - Based Argumentation Instructional Modules (E- 
BAIMs) that can be used in classrooms, and (2) teacher professional development focused on 
deep analysis of the E-BAIMs and implementations of them, thereby providing teachers with 
opportunities to develop the expertise needed to implement and adapt modules. 

Developing E-BAIMs is the work of design teams comprised of content specialists, teachers, 
educational researchers, and instructional developers. Development proceeds in iterative 
cycles of design-implementation-reflection-redesign. Teacher professional development 
occurs in the context of Teacher Network meetings. 

Purpose / Objective / Research Question / Focus of Study: 

In this symposium we report on lessons learned through design-based research regarding 
important features of the design and implementation of components of the E-BAIMs. 
Specifically, for the past two years, we have been working as three design teams - one for 
each disciplinary area - to construct initial versions of E-BAIMs that are tested iteratively in 
classrooms of teachers who are members of the design teams. The overarching research 
question is whether the intended outcomes of particular aspects of the design, as well as of 
the module overall, were achieved. More specifically: 

• To what extent were students engaged by the tasks and texts and showed evidence 
of learning? 

• To what extent were the designs of E-BAIMs supporting student dispositions 
(e.g., persistence, self-regulation) to engage in academic reading? 

• To what extent did teachers find the E-BAIMs feasible and useful? 
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• What modifications, improvements, and redesigns were suggested by teachers’ 
and students’ enactments of the E-BAIMs? 

Although not a major focus of this presentation, the design of the E-BAIMs is being 
influenced by basic studies on components of the designs. Likewise, the results of 
implementing the modules are raising questions that are taken up in basic studies. 

Initial designs of the E-BAIMs in each of the disciplines incorporated the same set of 14 
principles based on extant findings regarding reading and learning in literature, history, and 
science (e.g., Bransford, Brown, & Cocking, 2000; Goldman & Bisanz, 2002; Goldman, 
2005; Goldman, et ah, 2009; Greenleaf, et ah, 2011; Langer, 2010; Lee, 2011; Lee & 
Spratley, 2010; Pellegrino, Chudowsky, & Glaser, 2001;Shanahan & Shanahan, 2008; 
Wineburg, 2001). In synthesizing the research, we developed a broad framework that 
specified five categories of knowledge that constitute a model of the competent reader in 
each discipline: epistemology; inquiry practices/ways of reasoning; overarching concepts, 
themes, frameworks, big ideas; infonnation representations/types of texts; and discourse 
practices/language structures. Table 1 provides brief definitions of each of these (please 
insert Table 1 here). Although the categories apply to each discipline, their instantiation 
constitutes distinct reading models in each discipline. 

The design of the E-BAIMs is intended to engage students in addressing a problem, driving 
question, or issue that necessitates close reading of a set of texts typical of the discipline 
using inquiry strategies and involving understanding key concepts and principles of the 
discipline. Gateway and cultural modeling activities (Lee, 2007) are included in the modules 
to draw on the knowledge students bring to the classroom. Students engage one another in 
cognitive and metacognitive exchanges that support sense making and use of evidence to 
support claims. Teachers model ways of reading and analyzing texts as appropriate to the 
discipline. Various templates support synthesis and argumentation in oral and written modes. 
With each implementation we administered pre-post assessments on a different but related 
topic for history and science or using different literary works than those used in instruction. 
To date, we have implemented E-BAIMs in two topic areas within each of the three 
disciplinary areas in multiple grades (please insert Table 2 here). The specific topics and 
numbers of classrooms per grade span are provided in Table 2. For each designed module, 
we developed an analysis of the rationale and intended outcome of its inclusion with respect 
to development of specific content knowledge, core knowledge constructs, processes, or 
skills. For example, what texts were included in what sequence and why those texts and 
ordering. 

Setting and Population. E-BAIMs implementations have been conducted in classrooms in 
urban and exurban public or publicly-supported charter schools serving children from diverse 
backgrounds. Each classroom implementation shown in Table 2 reflects one teacher and a 
class of between 15-25 students. The teachers participated in the collaborative design of the 
modules that they implemented. 

Research Design 
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Method. Design-based research methodology was employed in studying each 
implementation, with each classroom constituting a case (Cobb, Confrey, diSessa, Lehrer, & 
Schauble, 2003; Yin, 2006). Two researchers were present each day of the implementations, 
taking field notes and videotaping. Interviews and debriefings were conducted with teachers 
on a daily basis. We intentionally designed the first and second iterations in science and 
history to occur over approximately 10 to 12, 50-minute class periods. The literature designs 
were intended to occur over 20 to 25, 50-minute class periods at the high school grade bands. 
The middle school literature design was intended to cover an 8-week span of instruction 
during the 90 - minute literacy block. 

Data Sources and Analytic Strategies. The implementations were documented via field 
notes, videotapes of the classroom implementations, interviews and debriefings with the 
teachers, interviews with students, and student artifacts produced as part of the E-BAIMs. 

Qualitative analyses of field notes, interviews, and analytic memos along with qualitative and 
quantitative analyses of the pre/post assessments and student artifacts over the course of the 
module were used to inform the fundamental research questions. We used the videos to more 
closely examine the implementation process itself. We segmented videos to focus on 
instructional episodes and coded for the focus of instruction: building prior knowledge of 
topics through gateway activities; prior knowledge of strategies through cultural data sets; 
supports for close reading; supports for and engagement in argumentation; argumentation as 
oral discourse and written argumentation, including claims - evidence - reasoning segments. 
We examined both dosage (amount of time) and quality of implementation of the design. 

Findings: Lessons Learned 

In this presentation we focus on what we have learned specifically in one disciplinary area, 
literature. Lessons learned from the first iteration informed redesign of specific elements in 
the second iteration. The lessons learned and redesign efforts were similar in history and 
science. 

First iteration pre-post test essays and student text annotations and written artifacts were 
analyzed with respect to several skills targeted in the intervention, including (a) making 
claims about the themes and the meaning of symbols in the stories, (b) providing evidence 
from the texts for those claims, (c) using literary reasoning strategies to connect claims and 
evidence, (d) comparisons and contrasts across texts. These analyses indicated increases in 
noticing symbols and stating claims and some increase in citing information from the texts 
but a lack of reasoning that connected the infonnation from the text to the claims. There was 
little evidence of comparison or contrast across stories. The analyses of the video from the 
interventions indicated that while students learned to employ heuristics for rejecting the 
literal and inferring the symbolic, there was little attention to the ways in which symbolic 
language shaped theme. As a consequence the focus on claims and evidence about themes in 
the stories tended to rest on appeals to personal experience, but not on the functions of 
rhetorical choices by authors. In addition, there was little instructional support for cross-text 
comparisons. Furthermore, teachers were short-circuiting close reading of the texts, in an 
effort as they reported “to get to the symbolism and thematic interpretations.” There were 
also insufficient supports in the instruction for writing arguments. 
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The findings from the first iteration led to some significant changes in the design of the 
second literature module, which was focused on unreliable narration. In the second design 
iteration, more instructional time was devoted to participation structures in which students 
took the lead in carrying out the problem solving work involved in interpreting literary 
works. This included systematic use of tools available in the E-BAIMs for close reading 
(e.g., annotation, prompts to guide the use of graphic organizers to keep track of plot, 
characters, and unusual events or things that were surprising or unexpected). Teachers 
redoubled their efforts to support students in close reading of the literary texts; they 
intentionally spent less time reading to students. Scaffolds in the form of sentence frames for 
argumentation were introduced to acquaint students with ways in which they could “word” 
their written claims and evidence, and use reasoning to relate the two. 

Similar issues were observed in the history and science implementations and similar 
modifications were made. For example, often at the first sign of students struggling with text, 
teachers tended to “jump in” and do the reading for the student or state the important points. 
Modifications to the instructional designs emphasize close reading for event chronology and 
explanatory models (history) or for causal chains from factors to outcomes (science) and 
provide graphic organizers to assist each. 

Conclusions and Implications for Design: 

Our initial rounds of design-based research enabled us to identify key areas that must be 
highlighted for both teachers and students. In particular, we have found that focusing 
teachers on assessing “who is doing the intellectual work?” during class helps them retain an 
explicit focus on supporting students to engage closely with texts - and to struggle to 
understand them. Other scaffolds such as sentence frames provide a useful toolkit for 
teachers to assist students comparing and contrasting across texts and expressing their 
comprehension and reasoning in written argument form. The results of our design-based 
research indicate several areas that we are pursuing in the development of the modules. We 
are (a) placing greater emphasis on specifying learning objectives and mechanisms for 
teachers to assess these during the course of a module, (b) developing “spotlight” lesson 
sequences that target specific elements in modules (e.g., developing criteria forjudging the 
quality of arguments, close reading for event chronology), (c) expanding the topics of 
modules, (d) pursuing ways to introduce core knowledge constructs and then deepen them 
over successive modules, (e) developing more nuanced ways of capturing text complexity, 
and (f) using basic studies to explore the affordances of various task interpretation, close 
reading, and argumentation scaffolds. Finally, we are expanding the sample of teachers who 
are contributing to the design and refinements of the E-BAIMs through our teacher 
professional development networks. 
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Appendix B. Tables and Figures 


able 1. Types of Knowledge 


Core Knowledge Constructs 

Brief Description 

Epistemology 

What counts as knowledge? How do we know what we know? 
Beliefs, values, and commitments that the reader draws upon 
to prioritize and warrant claims based on ...a conception of the 
terrain of literature;... the nature of science; the nature of 
history 

Inquiry practices/ways of 
reasoning 

Practices of reading and reasoning when doing historical, 
scientific, or literary inquiry. Practices of developing 
interpretations, explanations, and arguments. 

Overarching Concepts, themes, 
frameworks, big ideas 

Major unifying ideas and frameworks within a discipline 

Information 

Representations/Types of texts 

Various forms in which disciplinary information is conveyed 

Discourse and Language 
Structures 

Communication practices and structures established by a 
discipline 


Table 2. Implementations of E-BAIMs during October, 2011 to May, 2012 


Discipline & Topic 

Number of Classes per Grade Span 

Science: 

6-8 

9/10 

11/12 

MRSA (adaptation) 


2 


Human impacts on water purity 

1 



History: 




U.S. expansion and conflict with 
Native Americans 

1 


1 

Little Rock Nine & Civil Rights Era 

1 

1 

1 

Literary Texts: 




Modules on the theme of coming of 
age and understanding symbolism 

1 

2 

1 

Various themes with stories that 
involve unreliable narration 

1 

2 

1 
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